DUX-9411: Add Bedrock bedrock-mantle endpoint support for GPT-5.6 models - #21
Conversation
…fixed ids
reasoning_model? didn't match the "openai." vendor prefix Bedrock uses
(e.g. openai.gpt-5.6-sol), so encrypted-reasoning replay and the
temperature strip silently didn't kick in for those ids. parse_error
also returned a Hash instead of a message string for OpenAI-shaped
mantle error bodies ({"error": {"message": ...}}).
Co-authored-by: Sam Boland <[email protected]>
Generic plumbing for the upcoming bedrock-mantle protocol: Connection can now be pointed at a base URL other than the provider's default, and SigV4 signing can target a service namespace other than "bedrock". Both default to today's behavior, so every existing caller is unaffected. Co-authored-by: Sam Boland <[email protected]>
…e endpoint openai.gpt-5.x models (the GPT-5.6 Sol/Terra/Luna family) are reachable only via bedrock-mantle speaking the OpenAI Responses API — no Converse, no InvokeModel. MantleResponses subclasses Protocols::Responses, binds to a mantle-specific connection, picks /openai/v1/responses vs /v1/responses per model, and SigV4-signs against the bedrock-mantle service namespace instead of bedrock. Bedrock#protocol_for routes openai.gpt-5* ids to the new protocol ahead of the existing invoke_model/converse decision — deliberately narrower than a broad openai. prefix so gpt-oss (bedrock-runtime-native) keeps routing to Converse. #complete also skips the Converse-specific normalize_params on the mantle path so caller-supplied additionalModelRequestFields isn't forwarded to a Responses API call. Co-authored-by: Sam Boland <[email protected]>
models.dev doesn't have GPT-5.6 yet, and Bedrock.assume_models_exist? is false, so model resolution would raise without these. Mirrors the openai.gpt-5.5 entry shape (commit 0ffb655 set this precedent for claude-opus-5). Kept in its own commit so it's trivially droppable if these ids land upstream later. Pricing per million tokens: Sol 5.00/30.00 (cache_read 0.50), Terra 2.50/15.00 (0.25), Luna 1.00/6.00 (0.10) — cache_read at 90% off input in every tier. reasoning_options includes "max", which gpt-5.5 doesn't have. knowledge_cutoff is null; no date could be verified. Co-authored-by: Sam Boland <[email protected]>
|
Jira Issue: https://appfolio.atlassian.net/browse/DUX-9411 |
|
🤖 Supernova Code Review — View trace |
![staging-supernova-dx-appf-io[bot]](https://avatars.githubusercontent.com/in/2831657?s=60&v=4){kind=small}
There was a problem hiding this comment.
Reviewed the bedrock-mantle/GPT-5.6 changes. One actionable bug found: SigV4 signing for mantle requests hardcodes bedrock_region instead of the mantle_region the request host is actually built from, which breaks auth whenever bedrock_mantle_region differs from bedrock_region (see inline comment on auth.rb). Also left a minor duplication nit on the frontier-model regex. Everything else (protocol routing, connection base_url override, error parsing, reasoning-model regex fix, model/alias registry entries) looked correct and is well covered by the added specs.
| @@ -41,15 +41,15 @@ def sign_headers(method, path, body, base_url: api_base) | |||
| payload_hash | |||
| ].join("\n") | |||
|
|
|||
| credential_scope = "#{date_stamp}/#{bedrock_region}/bedrock/aws4_request" | |||
| credential_scope = "#{date_stamp}/#{bedrock_region}/#{service}/aws4_request" | |||
| string_to_sign = [ | |||
| 'AWS4-HMAC-SHA256', | |||
| amz_date, | |||
| credential_scope, | |||
| Digest::SHA256.hexdigest(canonical_request) | |||
| ].join("\n") | |||
|
|
|||
| signing_key = signing_key(date_stamp, credentials.secret_access_key) | |||
| signing_key = signing_key(date_stamp, credentials.secret_access_key, service) | |||
| signature = OpenSSL::HMAC.hexdigest('sha256', signing_key, string_to_sign) | |||
There was a problem hiding this comment.
sign_headers (and the signing_key helper it calls) hardcode bedrock_region in the SigV4 credential scope ("#{date_stamp}/#{bedrock_region}/#{service}/aws4_request") and in the region-derived signing key, with no way to pass a different region. But mantle_api_base is built from mantle_region, which can differ from bedrock_region via the supported bedrock_mantle_region config option (see bedrock.rb#mantle_region and the "prefers bedrock_mantle_region over bedrock_region when set" spec).
When an operator sets bedrock_mantle_region to a region different from bedrock_region, MantleResponses#mantle_signature_headers calls sign_headers(..., base_url: @provider.mantle_api_base, service: MANTLE_SIGNING_SERVICE) — the request goes to a host in mantle_region, but the SigV4 signature is computed using bedrock_region. AWS will reject this with a signature/region mismatch, so mantle requests break entirely for any config that actually exercises bedrock_mantle_region. None of the added specs cover a scenario where bedrock_mantle_region != bedrock_region, so this isn't caught.
Fix: thread the signing region through explicitly, e.g. add a region: keyword to sign_headers/signing_key (defaulting to bedrock_region for existing callers) and have mantle_signature_headers pass region: @provider.mantle_region.
There was a problem hiding this comment.
If this is still a problem, re-analyze and determine if it needs a fix.
There was a problem hiding this comment.
Already fixed — sign_headers/signing_key now take an explicit region: param (defaulting to bedrock_region), and MantleResponses#mantle_signature_headers passes region: @provider.mantle_region, so mantle requests sign correctly even when bedrock_mantle_region differs from bedrock_region. No further action needed here.
| # Frontier openai.gpt-5.x models are served at /openai/v1/responses; every other | ||
| # mantle model (e.g. openai.gpt-oss-*) is served at /v1/responses. See the | ||
| # AWS Bedrock model cards for GPT-5.6. | ||
| FRONTIER_GPT5_PATTERN = /\Aopenai\.gpt-5/ |
There was a problem hiding this comment.
nit: FRONTIER_GPT5_PATTERN here duplicates Bedrock::MANTLE_ONLY_MODEL_PATTERN (same regex, same intent). Since MantleResponses only ever handles models protocol_for already routed via mantle_only_model?, consider referencing Providers::Bedrock::MANTLE_ONLY_MODEL_PATTERN here instead of maintaining two copies that could silently diverge.
There was a problem hiding this comment.
Queued — will dedupe by reusing Bedrock::MANTLE_ONLY_MODEL_PATTERN instead of the separate FRONTIER_GPT5_PATTERN constant.
…ly works sign_headers hardcoded bedrock_region in both the credential scope and the signing key, even though mantle_api_base already derives its host from mantle_region. Any Sol-tier request (us-east-1/us-east-2) issued from a us-west-2-configured app would sign against the wrong region and get an InvalidSignatureException. Add a region: kwarg to sign_headers/signing_key, default to bedrock_region for full backward compatibility, and thread provider.mantle_region through MantleResponses. Also make mantle_region public so the protocol can call it, and extract the duplicated 'bedrock' default into DEFAULT_SIGNING_SERVICE. Adds a spec exercising a real signature with mismatched bedrock_region/ bedrock_mantle_region, and a spec asserting the signed X-Amz-Content-Sha256 matches a SHA-256 of the bytes WebMock actually received. Co-Authored-By: Claude Sonnet 5 <[email protected]> Co-authored-by: Sam Boland <[email protected]>
…istry provenance/ordering MANTLE_ONLY_MODEL_PATTERN (Bedrock) and FRONTIER_GPT5_PATTERN (MantleResponses) answer different questions but coincide today. Comment each pointing at the other so a future frontier family (e.g. gpt-6-*) doesn't get updated in only one place, silently 404ing or misrouting. Patterns themselves are unchanged. Also corrects the hand-added GPT-5.6 registry entries: "source": "models.dev" was false (models.dev has no gpt-5.6 yet, and that field gates what rake models preserves on a fetch failure) and "last_updated": "2026-06-01" predated the entries' own "created_at" of 2026-07-13. Reordered sol/terra/luna to alphabetical (luna, sol, terra) to match rake models' own sort and avoid a gratuitous reorder diff whenever it's next run. Co-Authored-By: Claude Sonnet 5 <[email protected]> Co-authored-by: Sam Boland <[email protected]>
…y params on mantle
Guard the new error.message dig against a String-valued `error` key (Mantle-style
{"error": "..."} bodies used to hit the body['error'] fallback pre-parse_error-change;
Hash#dig on a String raises). Also make strip_converse_only_params raise a clear
ArgumentError for top_k/additionalModelRequestFields on the mantle path instead of
silently forwarding them to a Responses API that would reject them with an opaque 400.
Add comments cross-referencing the two forward-looking/dead-branch spots flagged in
review: the bare `super` in Bedrock#complete forwards the reassigned `params`, and
MantleResponses#completion_url's non-frontier branch is currently unreachable in
production routing.
Co-authored-by: Sam Boland <[email protected]>
|
Double check that this isn't going to cause issues with exisitng code paths in any way. |
Confirmed, checked all call sites on the actual PR branch:
So this change is additive via keyword defaults and doesn't alter any existing non-mantle code path. |
2 participants
Summary
Add OpenAI (bedrock-mantle) endpoint support to ruby_llm, enabling access to frontier GPT-5.x models through AWS Bedrock's mantle routing protocol.
Changes
Core Support
lib/ruby_llm/protocols/mantle_responses.rb): Implements OpenAI Responses API compatibility for Bedrock's bedrock-mantle endpoint, with vendor-prefixed model ID handling and proper SigV4 signing region parameterizationlib/ruby_llm/providers/bedrock.rb): Adds mantle protocol registration, mantle endpoint configuration, and routing logic to discriminate between converse/invoke_model and mantle endpointslib/ruby_llm/providers/bedrock/auth.rb): Fixes signing region to usebedrock_mantle_regionwhen present, enabling cross-region bedrock-mantle requestslib/ruby_llm/connection.rb): Addsbase_url:parameter to Connection#initialize for protocol-specific endpoint routingModel Registry
lib/ruby_llm/models.json): Hand-added entries for GPT-5.6 Sol, Terra, and Luna with Bedrock-specific metadata (272K context, 128K output tokens)lib/ruby_llm/aliases.json): Canonical aliases mapping gpt-5.6-{sol,terra,luna} to bedrock vendor-prefixed IDsBug Fixes
lib/ruby_llm/protocols/responses/chat.rb): Fixed regex to properly match vendor-prefixed reasoning models (e.g.,openai.o1,openai.gpt-5) in addition to unprefixed patternslib/ruby_llm/providers/bedrock.rb): Added TypeError handling for non-JSON error bodies in mantle protocollib/ruby_llm/protocols/mantle_responses.rb): Rejects converse-only streaming parameters that bedrock-mantle endpoint doesn't supportTest Coverage
spec/ruby_llm/connection_base_url_spec.rb): Tests custom base_url routingspec/ruby_llm/protocols/mantle_responses_spec.rb): Comprehensive coverage of endpoint routing, error handling, and reasoning model supportspec/ruby_llm/models_gpt_5_6_bedrock_spec.rb): Verifies model registry and alias resolutionspec/ruby_llm/providers/bedrock_spec.rb): Extended with mantle routing and cross-region signing testsspec/ruby_llm/protocols/responses/chat_spec.rb): Added vendor-prefixed reasoning model detection testsRelated
Agent session: https://staging.supernova.dx.appf.io/coders/7f845d85-f6b4-4b5c-a2d1-9525fe86771f